In statistics, Fisher's scoring algorithm is a form of Newton's method used to solve maximum likelihood equations numerically.
Contents |
Let be random variables, independent and identically distributed with twice differentiable p.d.f. , and we wish to calculate the maximum likelihood estimator (M.L.E.) of . First, suppose we have a starting point for our algorithm , and consider a Taylor expansion of the score function, , about :
where
is the observed information matrix at . Now, setting , using that and rearranging gives us:
We therefore use the algorithm
and under certain regularity conditions, it can be shown that .
In practice, is usually replaced by , the Fisher information, thus giving us the Fisher Scoring Algorithm: